1. Field of the Invention
The present invention relates to an image processing apparatus and method for efficiently compressing and encoding still images and motion pictures. More particularly, it relates to a process for efficiently compressing and encoding motion pictures by adding a simple structure to an apparatus including a general-purpose digital camera integrated circuit (IC) having a function of compressing and encoding still images.
2. Description of the Related Art
An image processing apparatus 700 shown in FIG. 1 is an example of known compact portable equipment with a general-purpose digital camera IC (IC having a function of compressing still images to JPEG-format images (ISO/IEC 10918)) (JPEG is an abbreviation of joint photographic experts group).
Referring to FIG. 1, the image processing apparatus 700 includes a lens unit 701, a charge-coupled device (CCD) 702, a CCD control unit 703, a digital camera IC 704, a work memory 705, a key switch group 706, a NTSC output IC 708, and a display 709.
The lens unit 701 includes lenses having a fixed focus, zoom lenses having a predetermined magnification, and so on. The CCD 702 is a typical imager, the number of pixels of which is determined based on the resolution.
The CCD control unit 703 has a timing generator for supplying transfer clock and shutter signals to the CCD 702, a correlated double sampling/automatic gain control (CDS/AGC) circuit for eliminating noise and controlling gain for image signals from the CCD 702, an A/D converter for converting analog image signals into 10-bit digital data, and so on (these components are hereinafter also referred to as a CDS/AGC and AD converter).
Image data is constantly output from the CCD control unit 703 at a rate of 30 frames per second (fps).
The digital camera IC 704 performs image processing, such as white balance adjustment and control of the shutter speed and the aperture, on the image data from the CCD control unit 703 and converts the processed image data into digital data having a luminance (Y) and chrominance (Cb and Cr) format to output the digital data to the NTSC output IC 708. The digital camera IC 704 has a function of compressing captured image data to the JPEG-format data, which is then output to the NTSC output IC 708.
The work memory 705 for a work area required for the above processing and the key switch group 706 including a shutter switch, a mode switch, and the like are connected to the digital camera IC 704.
Since NTSC signals are generally output for display, the NTSC output IC 708 for outputting the signals to the display 709 serving as display means is connected to the digital camera IC 704.
The following two processes performed in the image processing apparatus 700 will now be described:                (1) a display output process with the digital camera IC 704 and        (2) a motion-picture recording process with the digital camera IC 704.(1) Display Output Process with the Digital Camera IC 704        
The following describes a case in which the image processing apparatus 700 is ordinarily set to a finder mode, that is, a mode in which images are only displayed on the display 709, without the shutter switch in the key switch group 706 being depressed.
First, the light reflected from an object is incident on the CCD 702 from the lens unit 701.
The CCD 702 converts the light into electrical signals in accordance with a timing specified by the timing generator in the CCD control unit 703 to output the converted signals to the CCD control unit 703.
The CDS/AGC and AD converter in the CCD control unit 703 converts the electrical signals from the CCD 702 into digital data to supply the converted data to the digital camera IC 704.
The digital camera IC 704 calculates parameters, such as AE, AF, and AWB, from the digital data (image data) supplied from the CCD control unit 703 to supply these new parameters to the CCD control unit 703.
The digital camera IC 704 processes image data in accordance with the parameter settings to compress the image data to a pixel size appropriate for display and outputs the compressed data to the NTSC output IC 708.
The digital camera IC 704, having a digital output function, ordinarily outputs YCbCr-format (4:2:2) digital data at a rate of 30 fps, one frame having 640×480 dots (a dot rate of 13.5 MHz).
Since an apparatus with an ordinary digital camera IC mainly uses display means conforming to the NTSC format, most digital camera ICs are constructed so as to not only output digital data but also display the NTSC-format image data.
Accordingly, the NTSC output IC 708 converts the NTSC signals from the digital camera IC 704 into the image data that is displayed on the display 709.
(2) Motion-picture Recording Process with the Digital Camera IC 704
When the shutter switch in the key switch group 706 is depressed while the image processing apparatus 700 is set to a motion-picture recording mode, the digital camera IC 704 compresses image data from the CCD control unit 703 into the JPEG format, generates a JPEG file and records it in the work memory 705, as in the finder mode described in section (1) Display Output Process with the Digital Camera IC 704.
The above processing is continuously performed while the shutter switch is being depressed. However, the capacity of the work memory 705 is limited. Therefore, limited specifications, such as the size of the image data to be recorded being not larger than the video graphics array (VGA) size (640×480 pixels), the frame rate being not higher than 15 fps, and the continuous photographing period being limited 30 seconds, are provided.
When the NTSC signals supplied from the image processing apparatus 700 (the output signals from the NTSC output IC 708) are input to a motion-picture capturing apparatus 800 in FIG. 2 with an MPEG4 IC including a coder/decoder (CODEC) conforming to the MPEG-4 standards (ISO/IEC 14496) (MPEG is an abbreviation of moving picture experts group), the motion-picture capturing apparatus 800 performs the following processing.
An encoding process conforming to the MPEG-4 standards is characterized in that motion pictures can be edited in encoded bit streams by encoding images for every object. Specifically, background images are encoded in accordance with a conventional encoding system for rectangular motion pictures, whereas foreground object images are encoded in accordance with an object-based encoding system. In the object-based encoding system, shape information, as well as luminance and color information, are encoded.
In the motion-picture capturing apparatus 800, an NTSC signal 801 input from the outside (the image processing apparatus 700) is supplied to an NTSC decoder 802.
The NTSC decoder 802 converts the NTSC signal 801 into YCbCr digital image data (4:2:2) conforming to the CCIR-656 and supplies the converted image data to an MPEG4 IC 803.
The MPEG4 IC 803 compresses the YCbCr digital image data from the NTSC decoder 802 in a work memory 804 under the control of a microcomputer 805.
Since a monitor outputs the YCbCr digital data (4:2:2) during compression processing and an MPEG-4 compressed file contains the YCbCr digital data, as for the input to the MPEG4 IC, an NTSC encoder 806 converts the YCbCr digital data into NTSC signals and supplies them as an NTSC output 807 to a television receiver or the like through an external terminal.
The microcomputer 805 controls, for example, the mode setting of the MPEG4 IC 803 based on key inputs by a user or the like.
In known compact portable equipment having a digital camera function, such as the image processing apparatus 700 in FIG. 1, when the motion pictures are compressed to a motion-JPEG format for recording, the amount of data per frame is very high owing to the low compression rate of the motion pictures. Hence, the resolution and the frame rate of the motion pictures are restricted because of the limited capacity of a work area or a detachable storage medium, such as a compact flash (CF) card or a secure digital (SD) card, and thus motion pictures can be captured only for a short period.
Accordingly, an image recording methods conforming to the MPEG-4 encoding system or the like are being adopted instead of the motion-JPEG image recording method. Since a known digital camera IC as shown in FIG. 1 is constructed so as to output only analog NTSC signals or an image file compressed to the JPEG format (JPEG file), it cannot produce the YCbCr-format image data required for the MPEG-4 compression.
Furthermore, a decoding IC only for the NTSC signals is necessary to generate the TCbCr-format image data from the NTSC signals.